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ABSTRACT 

A technique will be described for precise registration of two images of the same 
area, taken under different conditions. The images may differ in scale, orienta- 
tion, or aspect angle, they may have limited distortion, or they may be obtained 
from different sensors, or from maps. The technique, called AUTO-MATCH, in- 
volves digital preprocessing of the images to extract edge contours, followed by 
a correlation of corresponding edge end points. The availability of an array of 
endpoints makes possible subpixel registration accuracy. 

The technique is being applied to a system for assessment of geometric image 
quality (GIQAS), to be installed at NASA-Goddard later this year. This system 
will measure the registration vectors over an array of window pairs from Land- 
sat images, so as to determine the distortion between them. Similar measure- 
ments could be used to infer relative positioning of the spacecraft. The GIQAS 
system accuracy is specified at 0.2 pixels. 

The technique has also been considered for updating the inertial navigation sys- 
tems of missiles or aircraft. Scene matching between an image obtained aboard 
the vehicle and a stored reference can eliminate the drift of an inertial platform. 
In order to demonstrate this capability in real time, Westinghouse has assembled 
a laboratory demonstration of AUTO-MATCH, which will be described. 
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INTRODUCTION 


A digital scene matching technique will be described which performs 
precise registration between images under a variety of conditions. This 
capability can be used for geometric image correction, or for autonomous 
navigation of a vehicle with respect to terrain signatures. The technique 
involves two steps. First, the images are digitally preprocessed to 
extract edges. This is performed on a line-by-line basis in the images 
to achieve megapixel per second throughput rates. The edge end-points and 
its polarity (location of light and dark sides) are obtained, with a 
bandwidth compression of one or more orders of magnitude achieved. Second, 
edge end-points are compared between images. An algorithm is available 
which can perform registration between images independent of relative 
orientation, and of position and scale over wide ranges. Accuracy is 
possible to as Tow as 0.1 pixel depending on the available scene detail. 

Details of the digital image processor will be described first, 
followed by a description of the registration operation. Application of 
the technique to the NASA GIQAS system (Geometric Image Quality Assessment 
System) will then be considered, as well as the problem of autonomous 
vehicle navigation. 


DIGITAL IMAGE PREPROCESSOR 


Image registration operations are accomplished by a hybrid digital 
image processor, consisting of a special-purpose preprocessor (to extract 
key image data) followed by a programmable general-purpose processor (to 
extract registration vectors). The function of the preprocessor is to extract 
the features required for registration computations from the gray level 
image samples. These features are straight-line contours of density 
gradient. In effect, the preprocessor produces a line drawing of the image. 

At the same time, the data bandwidth is reduced by one or two orders of 
magnitude, depending on the selected thresholds for minimum gradient 
detection and line length. In addition, the preprocessor routinely 
extracts statistics associated with a uniform matrix of image areas. 

Operation of the preprocessor is on a line-by-line basis with respect 
to the input image. Therefore, video data may be handled directly. Further- 
more, storage requirements in the preprocessor are limited to single lines 
of data only. The preprocessor comprises a series of shift registers. 

Data flow through the line extraction portion of the preprocessor is shown in 
Figure 1. 

We will demonstrate the individual preprocessor operations with the use 
of Landsat images 1703-17590 (Reel 2) and 1739-17575 (Reel 2) representing 
location N 36 deg, W 120 deg (near Fresno, California) on June 26, 1974, 
and August 1, 1974, respeetively. Portions of these images (MSS Band 4) 
are displayed in Figure 2. In particular, we will concern ourselves 
with the windows A and A', measuring 125 x 125 pixels in size. Gray level 
density values for these two areas are shown by Figure 3. The numbers 
represent the density to the nearest of 16 levels. Numbers above 9 are 
indicated with an overprinted ’V". Large numbers represent dark areas. 
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Before preprocessing the images for line extraction, spatial filtering 
operations may be performed on the gray level data for purposes of noise 
smoothing or equalization of resolution in both dimensions. 

To generate the straight-line contours (subsets) of the image, it is 
first necessary to compute the two-dimensional gradient (derivative) at each 
image point. This is done with a four-pixel window, which scans across 
the image in a raster format. The gradient direction (0 to 360 degrees) 
is quantized to one of 16 discrete directions. To suppress the areas of 
negligible gradient activity (containing no significant contour or edge 
information), a threshold is applied to the gradient amplitude. Gradients 
with amplitudes less than the threshold are set at 0. A display of gradient 
directions for wi ndows A and A' in Figure 2 is shown in Figure 4. The 
numbers indicate direction from light to dark, (1 is from left to right) . 
Numbers above 9 are overprinted with a "/"• 

After gradient thresholding, the edges are generally still too wide for 
subset generation. Therefore, a gradient thinning operation is performed 
that basically "skeletonizes" wide edges . 

The al gori thm uses a raster scanning wi ndow containing a gradient cell , 
X, and four of its nearest neighbors. The neighbors with col inear gradient 
directions are compared to X. The largest gradient ampl i tude is then 
retained. This procedure is repeated sequentially for each gradient point 
in the image. The maximized gradient display for images A and A' is shown 
by Figure 5. Compare this with Figure 4. 

Edge generation is accomplished by "growing" a line formed by 
adjacent parallel gradients. A s before, a 5-cell scanning window is 
employed. The new data point is labeled cell X. Its four neighbors are 
examined to find those containing a parallel (within a tolerance) gradient 
direction. If one is found, then the neighbor is dropped as a line endpoint, 
and X becomes the new endpoint. Neighbors that are col inear to the gradient 
of X are excl uded to prevent false lines from forming. 

The edges derived from images A and A' are shown in the graphical 
plots in Figure 6. They are numbered consecutively according to their lowest 
position on the plots. The polarity ( 1 ocati on of light and dark sides) for 
each edge is coded by the presence or absence of stars at both ends. In 
general , the absence of the star indicates a change from light to dark in the 
clockwise direction, for an axis of rotation about the uppermost endpoint. 

For horizontal edges , the axis of rotation is placed at the right end 
of the subset. 

A second preprocessor function is the collection of statistics over 
a matrix of square image areas. The measurements amount to a textural 
analysis of thewi ndows. They include histograms of gray density, gradient 
amplitude and direction, and subset length. Average values for gray 
density and gradient amplitude are also computed. 


51 



INITIAL ACQUISITION 


The Westinghouse AUTO-MATCH techniques can achieve registration 
between two line-by-line scanned images of the same area totally independent 
of relative orientation and independent of position and scale over reasonable 
ranges. The registration output includes a measure of image similarity 
between the images, their relative scale and orientation, and the transformation 
of points between images. These steps are carried out in the sequence shown 
in Figure 7. 

With reference to Figure 7, video signals from both images are 
digitized and preprocessed to extract image density contours or edges, 
as described earlier. The edges are defined by their endpoints. Features 
are formed from the edges that consist of the geometric relationships between 
pairs of edges and provide properties that are invariant, regardless of 

relative position, orientation, or scale. For example, these properties 
can include the relative angles between edges. These features are compared 
between images, with the initial result being a measure of similarity asso- 
ciated with the registration, as well as a measure of their relative orienta- 
tion and scale. When the similarity match has been accomplished, endpoints 
on the corresponding contours in the images can be related. In particular, 
the center point of one image can be located on the second image. 

The computations associated with a pair of subsets are listed in 
Figure 8. Edges AB and CD in image A correspond to A'B' and CD' in image 8. 

A geometric feature is fonned between the pairs by connecting endpoints A 
and C and A' and C. The angles between the subsets and the connecting lines, 
Y 2 » t'i and Y '2 will be equal if figures ABDC and A'B'D'C are 

similar. Furthermore, their values are invariant with translation, scale, and 
orientation of these figures in the plane. Accordingly, these angles are 
the first elements in the geometric features used to achieve automatic 
acquisition. Along with these angles, the features also include the 
orientation, cj)', of the connecting lines relative to a reference axis 
in the image, and the lengths, S and S', of the connecting lines. They also 
include the edge endpoints. 

Initial acquisition is obtained by matching pairs of features between 
the images with respect to y» and by clustering the results with respect to 
relative orientation and relative scale. If the two images are identical 
except for translation, rotation, and scale, all the features will cluster 
at a point. If they have relative distortion, the cluster will be dispersed. 

The amount of acceptable distortion is controlled by an adjustable threshold. 

The center of the cluster provides values for the relative orientation 
and scale of the two images. These values may be used to determine a trans- 
formation between the x and y coordinates for the two images. Since corres- 
ponding features contain endpoint coordinates, each pair can be used to 
calculate the nominal coordinates of translation, x , y , between the images. 
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These are again clustered to obtain an average over the images, and to 
eliminate anomalies caused by symmetrical figures such as rectangles. 

The cluster center in the x , y. plane is used with the 4, S values to 
complete the final transformation from any point in image B to the corresponding 
point in image A. 

Where correction for image warp is desired registration is carried 
out on a matrix of windows having sufficient density to accommodate the 
warp function. 


PRECISION REGISTRATION 


In the event that relative orientation and scale between the images 
are known, as is often the case, a much simpler computation is possible. In 
this alternate computation, a search is made in one image for a companion 
to the endpoint found in the other. The search is carried out over a 
window corresponding in size to the positional uncertainties between 
matching endpoints. 

Either registration process differs from conventional point- by- point 
image correlation in several ways. First, only points Of high information 
density (endpoints) are used in the computations; so, the processing load is 
greatly reduced. Second, since the positions of the endpoints are individually 
available, the average offset between the image pairs can be estimated 
directly. Finally, since density values are not used, multispectral or 
multisensor registration can be accomplished. 

Registration operations that assume a prior knowledge of orientation 
and scale will be demonstrated on the pair of Landsat images in Figure 2. 
Preprocessor outputs for windows A and A' are given in Figure 6. The results 
obtained by stepping A against A' by 20 pixels in each direction are shown 
by Figures 9, 10, and 11. The match index contour of Figure 9 represents 
the number of matching endpoints at each location, divided by the number avail- 
able for matching at the same location. Although it is similar to a 
correlation coefficient, it is used only for the approximate location of 
the match point. The precise location of the match point is obtained from 
the zero crossing point for the Ax and Ay contours of Figures 10 and 11. Each 
point on these contours is obtained from the average offset for the matching 
endpoints in the x and y directions. A transition from negative to positive 
values is obtained, somewhat like a discriminator characteristic. 

Determination of the accuracy of the registration operation is based 
on the assumption that the uncertainty in the position of an endpoint, 
relative to its true position, is random. The uncertainty includes noise 
and quantization effects, and the characteristics of the preprocessor algorithm. 
If the average uncertainty is k' pixels, then the error for a window pair with 
n matching endpoints will be k'Vn. In a test with 40 pairs of Landsat images, 
it was found that the average error was less than 0.2 pixels, and the value 
for k for the Landsat images is approximately 2 pixels. 
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GEOMETRIC IMAGE QUALITY ASSESSMENT SYSTEM (GIQAS) 

The AUTO-MATCH teehnique is being applied to a system for assess- 
ment of geometric image quality, to be installed at NASA-Goddard later this 
year. The system will measure the registration vectors over an array of 
window pairs from LANDSAT images, so as to determine the distortion between 
them. A block diagram of the system, which will serve as one component 
of the Image Processing Facility (IPF), is shown by Figure 12. A real-time 
demonstration of registration performance, using a special-purpose preprocessor, 
is shown by Figure 13. The preprocessor is on the table at left. This unit 
provides data throughput rates up to 2.0 megapixels per second. Registration 
operations are carried out in the militarized minicomputer at center. 


AUTONOMOUS NAVIGATION 

The AUTO-MATCH technique could be used for precise autonomous navigation 
of spacecraft or aircraft in the manner shown by Figure 14. AUTO-MATCH 
makes use of a sensor on-board the vehicle, and provides update information 
to the vehicle navigation system. Reference data is obtained from previously 
sensed images or from ground operations. In either case, storage or 
transmission of the reference data can be accomplished with compressed 
bandwidth by first preprocessing the images. 

Westinghouse completed a feasibility study of the navigation update 
capability of AUTO-MATCH for the Air Force Avionics Laboratories in 1977. 

(Final Report AFAL-TR-77-88). A side-looking radar was proposed for the 
on-board sensor. In order to simplify reference data base preparation, US6S 
airphotos were used without modification. As a result of this choice, 
edge polarization data could not be used, since the relative contrast at 
the edges in radar and airphotos differs. Samples of the images used in 
the study are given in Figure 15. Four image pairs were correctly registered 
from these samples, in spite of heavy shadowing in the radar image. 

Differences in orientation and scale between samples were removed by simple 
computations applied only to the preprocessed edges, based upon estimated 
values for these parameters. The estimated values would normally be 
obtained from the navigation system. 

Performance statistics were obtained from a series of 58 radar/air- 
photo pairs covering mountainous and rural terrain, urban areas, and land- 
water combinations. For single image pairs, correct acquisition was obtained 
in about 90% of the samples. Fix reliability can be greatly improved by 
the use of the windows in redundant combinations. Fractional pixel position 
accuracy was also demonstrated on this same image set. 
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Figure 1, Flow of Data Through the Preprocessor 
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Figure 2. Portions of LANDSAT Images - (a) 1703-17590 and (b) 1739-17575 in the Vicinity 
of Fresno, California for June 26, 1974, and August 1, 1974, Respectively 
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Figure 3. Gray Level Density Values for Windows A and A’ in Figure 
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Figure 4. Gradient Directions for the Windows in Figure 
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Figure 5. Maximized Gradients for the Windows in Figure 
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Figure 6. Line Contours (Subsets) for the Windows in Figure 
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Figure 7. Registration Steps for Initial Acquisition 











Figure 8. Computations in the General Case of Initial Acquisition 
between Images with Uncertainty of Position, Orientation, and Scale 
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Relationship of Hardware Components for 
Geometric Image Quality Assessment System 
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Figure 13. AUTO-MATCH Demonstration in the 

Westinghouse Image Processing Laboratory 
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Figure 14. 







